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Abstract  —  Visual  analytics  are  a  growing  area  of  research  within  the  Department  of  Defense  that  targets  the 
effective  interleaving  of  analytical  reasoning  with  interactive  interfaces.  Capitalizing  on  the  human  capacity  for 
spatial  reasoning ,  visual  analytics  enhance  the  decision  maker ’s  understanding  of  the  underlying  decision  space  by 
augmenting  the  assimilation  of  complex  relationships.  Towards  this  end,  the  US  Army  Research  Laboratory  has 
developed  a  software  application  that  complements  traditional  social  network  analysis  and  allows  for  improved 
understanding  of  the  human  terrain.  The  software  application,  called  the  Heterogeneous  Data  Proximity  Tool 
(HDPT),  combines  a  data  dimensionality  reduction  routine  with  multidimensional  scaling  to  produce  an  analytic  to 
visually  interrogate  the  similarity  or  dissimilarity  among  a  given  set  of  high  valued  individuals.  The  HDPT  starts 
with  a  reference  data  set  representing  individuals  with  known  group  affiliations :  insurgents,  innocents,  and 
criminals.  As  intelligence  is  collected  about  new  individuals  within  an  area  of  operation,  HDPT  computes  and  plots 
their  relative  positions  with  regard  to  the  existing  reference  data.  The  resulting  analytic  portrays  the  relative 
position  of  the  new  individual  ’s  organizational  orientation  within  the  known  human  terrain.  This  paper  presents  the 
development  of  HDPT  and  the  results  from  its  participation  in  the  FY12  Product  Director  Command,  Control, 
Communications,  Computers,  Intelligence,  Surveillance,  Reconnaissance  (C4ISR)  &  Network  Modernization's  Event 
2012  (E 12)  field  exercise. 

Keyword:  Visual  Analytic,  Multidimensional  Scaling,  Dimensional  Reduction,  Situation  Awareness. 

I.  Introduction 

The  US  Army  Research  Laboratory’s  (ARL)  Tactical  Information  Fusion  Branch  has  as  its 
mission  the  research  and  development  of  advanced  information  analytics  to  assist  Soldiers  in 
determining,  utilizing  and  sharing  relevant  information  and  improving  the  synthesis  of  data  to 
decisions.  Challenging  these  efforts  are  not  only  the  unprecedented  increase  in  the  types  and 
amount  of  information  available,  but  the  human  judgment  necessary  to  evaluate  it  in  the  presence 
of  incomplete  and  inconsistent  information  within  time-critical  environments  [1,2].  Required  are 
innovated  methods  that  allow  the  efficient  and  effective  transformation  of  data  to  information  to 
knowledge.  One  process  proffered  to  tackle  this  challenge  is  visual  analytics. 

Visual  analytics  is  a  growing  area  of  research  within  the  Department  of  Defense  that  targets  the 
effective  interleaving  of  analytical  reasoning  with  interactive  interfaces.  Capitalizing  on  the 
human  capacity  for  spatial  reasoning,  visual  analytics  enhance  the  decision  maker’s 
understanding  of  the  underlying  decision  space  by  augmenting  the  assimilation  of  complex 
relationships  [3,4].  Visual  analytics  is  a  multidisciplinary  field  that  has  yield  significant  results  in 
an  array  of  paradigms  to  include  business,  medicine,  and  defense  [1,5]. 


Towards  this  end  and  the  subject  of  this  report,  the  Tactical  Information  Fusion  Branch  has 
developed  a  visual  analytic  application,  entitled  the  Heterogeneous  Data-reduction  Proximity 
Tool  (HDPT).  The  goal  of  the  HDPT  is  to  complement  traditional  social  network  analysis  by 
allowing  the  exploitation  of  a  social  network  based  upon  the  calculated  similarity  of  individuals 
(against  a  known  reference  set)  as  opposed  to  the  traditional  organizational  structure.  The  result 
is  an  improved  understanding  of  the  human  terrain  that  effectively  incorporates  qualitative  and 
quantitative  information  into  the  decision  making  process. 

This  report  documents  the  development  and  evaluation  of  the  HDPT  visual  analytic  application. 
In  Section  2,  background  information  on  the  statistical  approach  undertaken  and  the  rational  for 
choosing  the  particular  proximity  calculation  is  provided.  In  section  3,  the  system-level  design 
and  instantiation  of  the  HDPT  as  a  web  application  linked  to  the  Distributed  Common  Ground 
Systems  -  Army  (DSGS-A)  is  discussed.  The  results  of  from  HDPT  utilization  in  an  Army  field 
study  are  presented  in  section  4.  The  paper  concludes  with  lessons  learned  and  the  way  forward 
in  Section  5. 

II.  Background 

For  many  military  applications,  extracting  knowledge  from  high-dimensional  data  sets  is  a 
persistent  and  complicated  task.  This  is  especially  true  when  the  data  sets  are  of  mixed-data 
type,  wherein  the  attributes  defining  the  objects  to  be  compared  take  on  values  from  differing 
measurement  scales.  Moreover,  the  data  of  interest  are  typically  amorphous;  i.e.  not  linked  to  an 
explicit  theory  to  assist  the  researcher  in  making  inferences  or  predicting  structure.  For  these 
reasons,  HDPT  uses  multidimensional  scaling  (MDS)  for  visualizing  the  structure  of  the  data  and 
Gower’s  similarity  coefficient  as  the  algorithm  for  calculating  the  proximity  matrices.  The 
following  sections  provide  a  brief  background  on  both  MDS  and  Gower. 

A.  Multidimesional  Scaling 

Originating  out  of  the  fields  of  mathematical  psychology  and  social  sciences,  multidimensional 
scaling  (MDS)  is  a  data  analysis  approach  used  to  visually  interrogate  the  similarity  or 
dissimilarity  between  the  pair-wise  “distances”  among  a  given  set  of  objects  [6,7,8,9,10,11].  The 
values  of  the  distances,  sometimes  called  proximity  measures  or  similarity  measures,  can  be 
obtained  either  as  perceived  subjective  measures  or  calculated  objectively  within  the  pair-wise 
comparison  of  the  given  set  of  objects.  Most  often,  the  objects  are  vectors  of  the  form  X  =  (xi, 
X2,  xm)  with  the  components  Xk  known  as  attributes,  variables,  or  factors  collectively 
providing  the  basis  for  comparison  of  the  objects.  Given  a  similarity  matrix  for  a  set  of  objects, 
each  object  is  projected  as  a  point  in  n-space  -  arranged  so  the  distances  between  the  objects 
have  the  strongest  possible  relation  to  the  similarity  matrix.  The  intrinsic  power  of  MDS  is  that 
it  allows  the  dimensionality  reduction  of  a  complex  n-space  to  a  human  interpretable  2-  or  3- 
dimensional  space.  It  is  this  projection  that  promotes  the  exploratory  analysis  of  data’s  hidden 
structure. 

The  following  example  from  Forrest  Young’s  Understanding  Multidimensional  Scaling  [7]  uses 
the  mileage  between  10  American  cities,  shown  in  Table  1,  as  the  objective  similarity  measure. 
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Table  1:  Between-City  Mileage  Similarity  Matirix  [7] 

The  associated  MDS  2-D  visualization  output  would  appear  something  like  that  found  in  figure 
1.  Note:  the  geometric  model  allows  one  to  discern  the  underlying  structure  and  allow  human 
interpretation. 
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Figure  1:  MDS  US  City  Example  [7] 

Differing  from  other  forms  of  multivariate  statistics,  specifically  principal  component  analysis 
(PCA),  MDS  does  not  constrain  the  data  to  be  normally  distributed.  With  that  understanding,  it 
becomes  apparent  that  the  hidden  power  behind  meaningful  MDS  analysis  is  found  in  the 
construction  of  the  similarity  matrix  and  its  projection  into  the  reduced  space. 

B.  Gower  Similarity  Coefficient 

For  ideally  calculating  a  similarity  matrix,  all  of  the  defining  attributes  should  be  of  the  same 
data  type  [12],  Unfortunately,  for  many  real-world  problems  disparate  scales  of  measure  are 
common  place  making  matrix  calculation  problematic.  One  of  first  to  confront  the  combination 
of  quantitative  and  qualitative  (mixed  scales  of  measure)  was  John  Gower  [13].  Given  an  array 
of  objects  with  k  attributes  the  global  similarity  value  (Sij)  between  two  objects  is  defined  as  the 
summation  of  the  individual  attribute  similarities  (Sijk)  multiplied  by  a  possible  weighting  factor. 
Here,  sp  corresponds  to  the  measure  of  local  similarity  assigned  to  the  object  pair  (X;,  Xj), 
restricted  to  attribute  k.  The  summation  of  the  individual  similarities  is  divided  by  the 
summation  across  all  weights.  Gower’s  similarity  coefficient  equation  shown  below,  allows  for 
the  weighing  of  individual  attributes  and  the  possibility  of  missing  data. 
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The  classic  calculation  for  individual  similarities  is  given  below  -  where  Xj|(  and  Xjk  are  the  Xth 
attribute  for  objects  X;  and  Xj  respectively.  is  defined  as  the  range  for  that  particular 
quantitative  attribute.  In  recent  years,  numerous  extensions  to  similarity  measurement 
calculations  have  been  attempted  in  wide  array  of  subject  areas  from  image  processing  to 
medical  informatics.  Approaches  taken  include  but  are  not  limited  to  rough  sets  [14],  fuzzy  logic 
[15],  and  ordinal  extensions  [16]. 
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The  following  section  details  the  design  and  development  of  the  HDPT  visual  analytic 
application.  Specifically,  the  HDPT’s  development  for  concept  demonstration  in  the  US  Army’s 
Command,  Control,  Communication,  Computers,  Intelligence,  Surveillance  and  Reconnaissance 
(C4ISR)  On-The-  Move  (OTM)  2012  exercise  (E12)  and  associated  scenario. 


III.  HDPT  System  Overview 

While  there  exists  research  into  the  development  of  qualitative  and  quantitative  similarity 
analysis,  few  have  been  effectively  coupled  with  a  visualization  framework  and  fewer  still  have 
been  interactively  coupled  with  a  tactical  military  decision  support  tool.  Toward  that  end,  HDPT 
is  the  software  instantiation  of  a  visual  analytic  technique  that  effectively  combines  MDS  with 
the  flexibility  of  the  mixed-scale  Gower  similarity  calculation.  The  targeted  area  of  interest  for 
this  instantiation  is  the  assessment  of  individuals  within  a  tactical  social  network.  This  specific 
implementation  was  designed  as  a  concept  demonstration  for  the  Product  Director  Command, 
Control,  Communications,  Computers,  Intelligence,  Surveillance,  Reconnaissance  (C4ISR)  & 
Network  Modernization's  Event  2012  (E12)  exercise  and  was  integrated  with  the  US  Army’s 
Distributed  Common  Ground  System  -  Army  (DCGS-A)  program  as  data  feed. 

At  a  high  level  of  abstraction,  the  concept  of  operation  for  this  exercise  was  as  follows.  The 
HDPT  started  with  a  reference  data  set  that  represented  the  characteristics  of  individuals  with 
known  group  affiliations:  insurgents,  innocents,  and  criminals.  As  intelligence  was  collected 
about  new  individuals  within  an  area  of  operation,  HDPT  computed  their  similarity  to  the 
reference  data  set  and  plotted  their  relative  positions  in  an  associated  3-D  visualization  space. 
The  resulting  analytic  portrayed  the  relative  position  of  the  new  individual’s  orientation  within 
the  known  human  terrain  (insurgent  vs  innocent  vs  criminal).  The  goal  of  the  tool  is  to  give  a 
military  analyst  improved  understanding  of  the  local  human  environment  and  assist  in  defining 
future  information  requests. 


Shown  in  Figure  2  are  three  major  components  that  make  up  the  HDPT  system: 

■  the  HDPT  Web  Application, 

■  the  DCGS-A  Global  Graph,  and 

■  the  statistics  engine. 


+ 


Global  Graph 


Confimunicate  Hmvttb 
str  \  arth  and  Upd  Jte] 
using  ISON  Formal 


_ ± 

Apache  Tomcat  Web  Server 


Ozone  Widget  Fra  mework 

II 

HDPT  Web  Application 

_ 

HDPT 


Communicate  with  Rserwe 
a  TCP/IP  «rwr  allowing 
mnotefonnettion  to  R, 
for  itaUstits  emulations. 


Rserve 

R 

L 

_ 

Statistics  Engine 


Figure  2:  HDPT  System  Diagram 

HDPT  is  a  web  application  that  is  deployable  within  the  Ozone  Widget  Framework  (OWF).  It 
accesses  the  DCGS-A  Global  Graph  via  web  service  as  its  data  source.  To  calculate  similarity  it 
uses  the  Gower  Similarity  and  Multidimensional  Scaling  algorithms  contained  in  the  R  statistical 
computing  environment.  The  Global  Graph  and  OWF  are  both  products  by  Potomac  Fusion, 
Inc.,  and  part  of  the  DCGS-A  program  of  record  (POR).  The  principal  HDPT  component  and 
subject  of  the  remainder  of  this  paper  is  the  HDPT  Web  Application.  The  following  subsections 
outline  the  design  of  the  HDPT  Web  Application  as  demonstrated  at  the  C4ISR  E12exercise. 

A.  HDPT  Web  Application 

The  HDPT  web  application  is  the  core  of  the  HDPT  system  and  serves  as  the  user  interface  for 
performing  similarity  analysis.  The  primary  components  of  the  HDPT  web  application  used  in 
E12  are  displayed  in  Figure  3  and  include  the  following:  (1)  HDPT  Menu  Bar,  (2)  Search 
Window  Panel,  and  (3)  Plot  Window  Panel. 
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Figure  3:  HDPT  Web  Application  Components:  (1)  Menu  Bar  (2)  Search  Window  (3)  Plot  Window 


1 )  HDPT  Menu  Bar 

The  HDPT  Menu  Bar  contains  drop-down  selections  for  loading  data,  creating  the  visual  analytic 
and  configuring  the  HDPT.  The  menu  bar  allows  access  to  the  Data  Menu ,  Analysis  Menu ,  and 
the  configuration  components. 

The  Data  Menu  contains  drop-down  selections  for  loading  data  into  the  tool 
and  propagating  updates  back  to  the  data  source.  The  data  source  used  by 
HDPT  during  the  E12  exercise  was  the  DCGS-A  Global  Graph.  In  this 
exercise,  an  SQL  version  of  the  Global  Graph  was  used  that  consisted  of  a 
PostgreSQL  database  and  associated  web  services  for  searching  and  updating 
the  database.  A  REST  (Representational  State  Transfer)  web  service  protocol 
was  used  for  communication  between  HDPT  and  the  data  source  via  a  JSON 
(JavaScript  Object  Notation)  data  structure.  REST  is  a  lightweight  alternative 
protocol  to  mechanisms  like  SOAP,  RPC  or  COBRA.  Typically  with  REST, 
simple  HTTP  is  used  to  make  the  connections.  Likewise,  JSON  is  a 
lightweight  data-interchange  format  designed  for  exchanging  human-readable 
structured  text.  The  selection  of  these  protocols  greatly  facilitated  connection 
and  interaction  with  the  DCGS-A  framework. 


The  Analysis  menu  contains  drop-down  selections  for  plotting  the  visual 
analytic  as  an  interactive  3D  scatter  plot  and  setting  user  preferences.  The 
Plot  menu  action  becomes  available  once  there  is  an  active  search  window 
panel.  In  addition  to  plotting,  the  analysis  menu  allows  two  user  preferences 
to  be  modified:  links  displayed  and  attribute  threshold.  Both  of  these 
preferences  change  the  way  links  between  nodes  in  the  plot  are  handled.  By 
default,  when  interacting  with  the  plot  the  user  can  right  click  on  a  node  and 
links  will  be  drawn  to  the  three  most  similar  reference  nodes;  the  number  of 
links  drawn  can  be  increased  or  decreased. 

-  The  configuration  menu,  allows  the  user  to  specify  server  parameters  for  the 
Global  Graph  and  Rserve.  There  is  a  dropdown  menu  for  each  server 
selection  that  contains  many  commonly  used  servers  for  the  OTM  exercise. 

In  addition,  mandatory  fields  are  provided  for  users  to  enter  their  username 
and  password  for  Global  Graph  access.  HDPT  uses  these  values  for 
connecting  to  both  the  Global  Graph  REST  web  services  and  the  R  statistics 
engine. 

2 )  Search  Window  Panel 

The  HDPT  search  window  panel  displays  the  dataset  returned  from  a  Global  Graph  search,  as 
seen  in  Figure  4. 


R  Global  GraDh  Person  Search  n*  13*  13 

Name 

Color 

ID 

Tribal  Affiliat. 

Aqe 

Gender 

Marital  Status 

Aali  Abu  Bakr  Karim 

BLACK 

11 

NA 

39 

MALE 

NA  1 

Abu  Navid  Sultan 

BLACK 

32 

PASHTU 

42 

MALE 

NA  1 

Amirah  Sani  El-Amin 

BUCK 

36 

NA 

48 

FEMALE 

NA  1 

Habib  Ala  Ahmed 

BUCK 

34 

NA 

23 

MALE 

NA  1 

Harun  Shahzad  El-Mofty 

BUCK 

9 

'na 

46 

MALE 

NA  1 

Hussain  Mansoor  El-Hashem 

BUCK 

22 

NA 

32 

MALE 

NA  1 

Ikram  Itimad  Abdullah 

BUCK 

16 

HAZARA 

20 

MALE 

MARRIED  1 

Khalilah  Qismat  Amirmoez 

BUCK 

13 

NA 

25 

FEMALE 

NA  1 

Rasul  Anass  Zaman 

BUCK 

4 

NA 

31 

MALE 

NA  1 

Rasul  Zayn  Mohammed 

BUCK 

20 

NA 

55 

MALE 

NA  1 

Ridha  Mahdi  El-Mofty 

BUCK 

19 

NA 

30 

FEMALE 

NA  1 

Saif-al-Din  Jinan  Hakim 

BUCK 

24 

NA 

50 

MALE 

NA  1 

Sami  Mis'id  El-Ghazzawy 

BUCK 

5 

NA 

38 

MALE 

NA  1 

Yusuf  Mehmud  Samara 

BUCK 

15 

NA 

42 

MALE 

NA  1 

Zaman  Noor  Hakim 

BUCK 

7 

NA 

34 

MALE 

NA  1 

Bahij  As’ad  Tawfeek 

GREEN 

6 

PASHTU 

28 

MALE 

MARRIED  / 

Farouk  Ghayth  El-Ghazzawy 

GREEN 

26 

PASHTU 

30 

MALE 

MARRIED  1 

Gabr  Hussein  Ahmed 

GREEN 

1 

BALOCH 

35 

MALE 

SINGLE  i 

HashimFouadAdmad 

GREEN 

3 

TAJIK 

35 

MALE 

SINGLE  / 

Jalal  Anas  Kader 

GREEN 

18 

PASHTU 

59 

MALE 

SINGLE  / 

Nazli  Gauhar  Ajam 

GREEN 

8 

PASHTU 

34 

FEMALE 

MARRIED  / 

Rafiq  Saif-al-Din  Karim 

GREEN 

33 

PASHTU 

21 

MALE 

SINGLE  / 

Shakira  Nashwa  Abujamal 

GREEN 

37 

PASHTU 

37 

FEMALE 

MARRIED  / 

Haroun  Salih  Abdullah 

ORANGE 

39 

TAJIK 

41 

MALE 

SINGLE  1 

Jinan  Qadir  El-Ghazzawy 

ORANGE 

27 

BALOCH 

44 

MALE 

SINGLE  / 

Karam  Imen  Boulos 

ORANGE 

29 

PASHTU 

31 

MALE 

SINGLE  1 

Majid  Rusul  Abujamal 

ORANGE 

25 

TAJIK 

38 

MALE 

SINGLE  1 

Mostafa  Tufayl  Karimi 

ORANGE 

28 

PASHTU 

30 

MALE 

SINGLE  1 

Najwa  Nadia  Saqqaf 

ORANGE 

17 

HAZARA 

40 

FEMALE 

SINGLE  1 

Rana  Lubna  Samara 

ORANGE 

12 

HAZARA 

43 

FEMALE 

SINGLE  1 

Salim  Mus'ad  Hakim 

ORANGE 

2 

PASHTU 

27 

MALE 

SINGLE  / 

Adam  Abdur-Rashid  Abdullah 

RED 

10 

BALOCH 

35 

MALE 

MARRIED  1 

Harun  Ziad  Boulos 

RED 

14 

TAJIK 

41 

MALE 

MARRIED  1 

Miraj  Rashid  Karimi 

RED 

21 

TAJIK 

29 

MALE 

MARRIED  1 

Mostafa  Farooq  Darzi 

RED 

38 

PASHTU 

54 

MALE 

MARRIED  / 

NasirBaki  Saab 

RED 

23 

HAZARA 

34 

MALE 

SINGLE  1 

Sulaiman  Badr  Muhammad 

RED 

31 

PASHTU 

44 

FEMALE 

SINGLE  / 

Wafi  Murtada  Hakim 

RED 

30 

BALOCH 

50 

MALE 

MARRIED  / 

Ziyad  Guda  Sultan 

RED 

35 

HAZARA 

34 

MALE 

MARRIED  1 

Figure  4:  Search  Window  Panel 


The  window  consists  of  a  table  where  each  row  is  a  person  and  each  column  is  an  attribute.  The 
result  returned  from  the  Global  Graph  web  search  service  is  a  JSON  structure  containing  all 
matching  people.  HDPT  processes  the  JSON  structure  into  a  separate  Java  object  for  each 
person.  The  Search  window  has  a  limited  number  of  user  interactive  features.  In  addition  to 
attribute  editing,  which  was  discussed  previously  as  part  of  the  Analysis  Menu  actions,  the 
Search  Window  panel  allows  rows  to  be  sorted  according  to  the  values  in  any  column.  Clicking 
the  mouse  on  the  column  heading  will  cause  the  rows  to  sort  alphabetically  (words)  or  number 


order  (digits)  according  to  the  data  in  that  column.  For  example,  the  search  panel  shown  in 
Figure  4  is  sorted  by  the  “Color”  column.  Finally,  if  the  plot  window  is  currently  active,  clicking 
on  a  row  in  the  search  window  will  highlight  the  node  that  corresponds  to  that  row  in  the  3D 
scatter  plot;  changing  that  node’s  color  to  a  neutral  yellow. 

3)  Plot  Window  Panel 

The  Plot  Window  Panel  displays  a  3D  scatter  plot  of  MDS  results.  The  development  of  the 
visualization  used  the  JMathPlot  open  source  graphics  library.  A  sample  Plot  Window  Panel 
from  the  E12  exercise  is  shown  in  figure  5.  In  this  case,  the  node  under  investigation  is  high¬ 
lighted  using  a  neutral  yellow  color  and  has  the  three  ‘most’  similar  reference  nodes  linked  with 
straight  lines;  sharing  similarity  between  two  nodes  from  the  criminal  set  (orange  color)  and  one 
from  the  friendly  set  (green). 


Figure  5:  Plot  Window  Panel  showing  Similarity  Links 

B.  HDPT  Plot  Window  Panel  Utility 

In  the  El 2  exercise  the  nodes  were  color  coded  with  the  following  population  schema: 
BLACK=Unknown,  RED=Insurgent,  GREEN=Friendly,  and  ORANGE=Criminal.  As  the 
exercise  transpired  data  describing  Unknown  persons  was  updated  in  the  Global  Graph,  either 
from  the  HDPT  console  or  a  mobile  device  ARL  developed  for  soldiers’  use  during  field 
encounters.  As  the  data  associated  with  each  BLACK  node  changed,  the  HDPT  visual  analytic 
also  changed.  Soldiers  participating  in  the  exercise  as  Intelligence  Analysts  were  tasked  with 
using  the  scatter  plot  to  determine  the  disposition  of  BLACK  nodes.  That  is,  did  projection  of 
the  dimensionally  reduced  characteristics  of  each  BLACK  node  more  likely  belong  to  the  RED, 
GREEN,  or  ORANGE  population  as  determined  by  proximity  (similarity). 

To  assist  the  Soldier’s  understanding  of  the  underlying  decision  space,  HDPT  provided  a  couple 
of  important  capabilities  to  the  3-D  visual  analytic  that  were  used  extensively  throughout  the  E 12 


exercise.  First,  as  shown  in  figures  6(a)  and  6  (b),  HDPT  provided  the  ability  to  freely  rotate  the 
decision  space  along  any  axis.  Projection  of  a  3-D  decision  space  onto  a  2-D  screen  can  be 
problematic;  objects  that  appear  close  to  one  another  in  2-D  can  actually  be  far  apart.  The  ability 
to  rotate  along  any  axis  was  critical  to  correctly  interpreting  the  relation  projections  of  the 
decision  space. 
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Figure  6(a)  View  before  rotation 


Figure  6(b)  HDPT  Rotated  View 


A  second  capability  that  was  utilized  at  length  was  the  zoom.  As  shown  in  figure  7(a)  and  7(b), 
the  zoom  capability  permitted  users  the  ability  to  examine  in  finer  detail  the  related  nodes  that 
were  clustering  close  to  the  node  in  question. 


Figure  7(a)  View  before  zoom 


Figure  7(b)  HDPT  zoomed  View 


In  this  example,  the  node  in  question  (yellow  node)  shows  a  high  similarity  to  (3)  neighboring 
nodes;  one  from  each  of  the  categorical  types. 

The  Plot  Window  Panel  also  possessed  several  interactive  features  to  assist  in  exploratory 
analysis.  Left  clicking  on  a  node  in  the  display  both  highlighted  it  and  the  corresponding  row  in 
the  search  window  so  that  the  node  is  referenced  back  to  its  original  data.  Second,  as  discussed 
previously,  right  clicking  on  a  node  will  draw  links  from  itself  to  3  or  more  of  its  most  similar 
reference  node  neighbors. 

IV.  HDPT  C4ISR  -E12  Concept  Evaluation 

The  goal  of  the  C4ISR  El 2  event  is  to  provide  stakeholders  from  across  the  Department  of 
Defense  assess  to  next-generation  technologies.  The  annual  event,  held  at  Fort  Dix,  New  Jersey, 
offers  researchers  a  military  relevant  venue  to  assess,  evaluate  and  validate  emerging 
technologies  and  facilitate  technology  maturation  and  transition  to  the  acquisition  process.  The 
following  section  details  HDPT  participation  at  the  E12  event. 

A.  Reference  Data  Set  Description 

The  HDPT  team  created  a  multiple-dimensioned  attribute 
data  set  describing  persons  affiliated  with  several  groups 
presumed  to  be  operational  in  the  El 2  exercise  area.  The 
groups  included  personnel  considered  to  be  non-hostile  or 
friendly,  overtly  hostile  (insurgent  terrorists),  and  locally 
hostile  (criminal).  The  attribute  set,  shown  in  Table  2, 
represents  information  collectable  through  combat 
questioning  of  individuals  met  during  field  encounters.  To 
support  HDPT  analysis,  each  attribute  can  take  the  form  of 
one  or  more  specific  values.  For  instance,  the  Martial  Status 
attribute  has  a  value  of  either  Married  or  Not  Married  to 
represent  the  current  condition  of  an  individual.  A  full  set 
of  specific  values  for  a  person’s  attributes  constitutes  a 
unique  data  set  representing  that  specific  individual.  For 
illustrative  purposes,  an  example  of  a  person  data  element  set  is  shown  in  Figure  8.  The  subject’s 
name  is  Bahij  As’ad  Tawfeek,  a  male  subject.  He  is  28  years  old,  married  and  a  member  of  the 
Pastun  tribe.  Bahji  was  born  in  the  area  and  is  an  afghan  national  (HN  is  Host  Nation).  Soldiers 
have  observed  him  wearing  a  military-like  uniform  and  have  associated  him  with  a  burgundy 
luxury  sedan.  He  has  no  criminal  record,  is  highly  educated,  and  is  considered  to  be  holding 
down  a  white  collar  job  as  a  mechanic  serving  in  the  local  military.  He  belongs  to  a  milder 
religious  sect  where  he  lives  in  Time  Square  Village. 


Table  2:  HDPT  Data  Attributes 

Tribal 

Affiliation 

Education 

Level 

Marital  Status 

Employment 

Type 

Nationality 

Military  Record 

Place  of  Birth 

(person 
religion) 
Member  of 

EquipmentID 

Skill 

VehiclelD 

AddressID 

Criminal 

Record 

Name 

SubID 

Location 

Org 

Age 

Gen 

TA 

MS 

Nat 

POB 

Equip 

VehiclelD 

CR 

Ed 

Emp 

MR 

Rel 

Sk 

Add 

Bahij  As'ad  Tawfeek 

39.98/ -74.43 

Friendly 

28 

M 

Pastun 

M 

HN 

BIA 

Uniform 

Burgundy  Luxury  Sedan 

No 

High 

WC 

ss 

Mid 

ME 

TSV 

Figure  8:  Example  of  a  person  data  element  set 


For  the  purposes  of  testing  HDPT,  the  ARL  team  created  thirty-nine  attribute  sets,  similar  to 
Figure  8.  These  were  broken  down  into  three  reference  groups  each  containing  eight  individuals 
representing  the  friendly,  insurgent,  and  criminal  organizations  and  fifteen  sets  representing 
initially  unknown  individuals.  The  reference  groups  formed  node  clusters  in  the  HDPT  visual 
plot,  with  each  node  representing  one  person.  Figure  9  shows  the  HDPT  display  with  both  the 
insurgent  (red  nodes)  and  the  friendly  (green  nodes)  organizations  circled.  The  black  nodes 
represent  unknown  individuals.  For  ground  truth  purposes,  the  full  set  of  attributes  for  each  of 
the  fifteen  unknown  persons  placed  their  representative  black  nodes  within  an  HDPT  reference 
cluster.  The  removal  of  data  from  the  thirteen  fields  listed  in  Table  2  causes  the  neutral 
alignment  of  black  nodes  for  the  initial  HDPT  display  shown  in  Figure  9. 


Figure  9:  HDPT  Reference  Group  Visual  Display. 

B.  Scenario  Inputs 

During  each  day  of  the  field  exercise,  Soldiers  conducted  a  checkpoint  and  a  presence  patrol.  It 
was  during  these  missions  that  combat  questioning  of  up  to  four  threat  actors  (unknown 
individuals)  was  performed.  The  questioning  was  used  to  obtain  data  to  populate  the  character’s 
thirteen  missing  fields.  Early  in  the  exercise,  the  ARL  team  found  that  the  realistic  conditions  of 
combat  questioning  most  often  yielded  only  a  part  of  the  data  necessary  for  full  analysis.  To 
support  a  thorough  study  of  HDPT  capabilities,  the  ARL  team  injected  data  for  eight  unused 
characters  to  ensure  that  an  optimal  set  of  attributes  was  available  for  analysis  by  soldiers 
performing  the  military  intelligence  functions  within  the  tactical  operations  center.  Four  of  the 
characters  were  represented  using  the  complete  set  of  thirteen  data  attributes  while  the  other  four 
characters  had  data  for  only  six  attributes.  The  resulting  data  mix  provided  an  ad  hoc  condition 
from  the  field  data  as  well  as  a  controlled  condition  supportive  of  HDPT  ground  truth  analysis. 


Data  injects  were  divided  into  two  sections:  one  for  the  morning  mission  and  one  for  the 
afternoon  mission.  Figure  10  shows  an  example  visual  analytic  of  a  daily  progression  of  data 
found  for  a  single  character.  The  character  node  is  colored  yellow  and  has  the  three  most  similar 
nodes  linked  with  straight  lines.  The  left  visual  analytic  represents  data  discovered  in  the 
morning  of  an  exercise  day  while  the  visual  analytic  on  the  right  is  an  accumulation  of  the 
morning  data  and  data  discovered  during  the  afternoon.  The  progression  of  data  discovery  is 
representative  of  intelligence  accumulation  on  persons  of  interest  over  a  period  of  time.  In  this 
example,  the  morning  data  causes  the  character  to  appear  friendly,  but  additional  data  from 
afternoon  collection  casts  the  character  more  as  a  criminal.  The  ground  truth  puts  this  character 
in  the  criminal  organizational  group. 


Figure  10:  Progressive  data  discovery  for  a  single  unaligned  person 


C.  Soldier  Survey 

To  track  the  progression  of  the  soldiers’  understanding  throughout  an  exercise  day  a  survey  form 
(see  appendix  A)  was  used.  Using  the  HDPT  visual  analytic,  each  Soldier  was  asked  to  give  an 
initial,  mid-morning  and  mid-afternoon  assignment  for  each  of  the  unknown  (black  nodes) 
individuals  association  to  a  group  and  to  weight  that  assignment  on  a  scale  from  1  to  5.  The 
Soldiers’  assignment  as  compared  to  character  ground  truth  was  used  as  a  measure  of  HDPT’s 
utility.  Soldiers  were  also  given  the  chance  to  comment  on  their  understanding  of  character 
situational  awareness  based  on  the  HDPT  visual  analytic,  the  utility  of  the  tool  and  open 
comments  for  improvements. 

D.  Results 

During  the  exercise,  there  were  11  different  soldiers  acting  as  intelligence  analysts  using  the 
HDPT.  Most  of  these  soldiers,  10  out  of  the  11,  were  infantryman;  typical  of  those  found  on  a 
Company  Intelligence  Support  Team.  Few,  2  out  of  the  11,  had  military  intelligence  training; 
while  most  felt  confident  in  their  computer  use  abilities.  Using  a  likert  scale  of  1  to  5  (where  1 


equated  to  very  poor  and  5  equaled  very  well )  the  survey  of  the  Soldiers’  revealed  the  following 
concerning  the  use  of  the  HDPT,  the  effectiveness  of  HDPT  to  track  the  evolving  High  Value 
Individual  (HVI)  intelligence  picture  and  their  opinion  of  the  usefulness  of  HDPT  in  a  tactical 
deployment.  Most  found  that  HDPT  was  easy  to  use;  scoring  usability  a  4.18  out  of  a  possible  5. 
The  soldiers  also  thought  that  HDPT  addresses  a  tactically  useful  function;  scoring  a  4.09  out  of 
a  possible  5.  Some  soldiers  were  concerned  about  HDPT’s  ability  to  provide  an  improved 
understanding  of  civilian  personnel  within  the  exercise  area  of  operations;  scoring  a  3.8  or  a 
possible  5.  Concerns  raised  during  the  El 2  event  were  reflected  in  the  comments  section  of  the 
survey. 


When  commenting  on  the  HDPT,  soldiers  suggested  the  development  of  several  features  that 
were  implemented  during  the  exercise,  these  included:  connection  of  the  three  persons  closest  to 
an  unknown  node  under  examination  with  straight  line  upon  querying  and  an  improved 
resolution  of  nodes  that  overlap  in  the  HDPT  visual  analytic.  Additionally,  soldiers  suggested 
several  improvements  are  being  developed  currently,  to  include:  highlighting  of  the  HDPT 
person  data  spreadsheet  corresponding  to  both  a  chosen  node  and  the  three  closest  nodes  to  that 
node  and  visual  analytic  rotation  while  in  a  zoomed  portion  of  the  visual  analytic.  As  these 
suggestions  originate  from  the  ultimate  field  user  of  the  HDPT,  each  will  strengthen  the 
relevance  of  the  final  product.  The  value  of  such  interactions  early  within  the  technology 
creation  process  is  crucial  to  ensure  the  creation  of  the  highest  quality  tools  for  our  soldiers’  use. 
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Table  3:  HDPT  Survey  Result  Table 


Soldiers  using  the  HDPT  examined  the  similarity  of  each  unknown  node  with  two  goals:  first,  to 
predict  in  which  reference  group  an  unknown  node  belonged,  and  second,  to  provide  a 
confidence  on  that  prediction  with  strength  of  from  1  (least  confident)  to  5  (most  confident). 
There  were  two  different  soldiers  using  HDPT  during  each  day  of  the  exercise.  As  shown  in 
Figure  11,  the  Soldiers’  analysis  of  nodes  with  both  full  and  partial  attribute  data  resulted  in  a 
high  percentage  of  correct  predictions.  In  fact,  as  determined  the  strength  coefficient,  most 
analysis  became  progressively  more  correct  or  remained  at  a  high  level  of  correctness  throughout 


an  exercise  day.  Using  the  HDPT  visual  analytic  at  the  end  of  exercise  days,  soldiers  correctly 
predicted  a  node’s  ground  truth  with  a  93%  accuracy. 


Data  Analysis  Results  (All  Person  Attributes) 


Recording  Period 


Figure  1 1 :  Soldier  predictive  analysis  using  the 


Data  Analysis  Results  (Partial  Attributes) 


Analysis  Recording 

visual  analytic  as  compared  to  ground  truth. 


V.  Conclusion 

Exploratory  data  analysis,  cluster  analysis,  pattern  recognition,  data  fusion,  data  mining  ...all 
benefit  from  visual  acuity  for  the  most  lucid  knowledge  extraction  from  high-dimensional  data 
representations.  MDS  is  a  powerful  technique  holding  potential  to  contribute  in  all  of  these 
areas.  Toward  that  end,  HDPT  was  developed  to  demonstrate  the  utility  of  similarity  analysis 
via  a  visual  analytic  in  understanding  the  human  terrain.  The  deployment  of  HDPT  in  a  tactical 
environment  (Ozone  Widget  and  Global  Graph)  made  it  possible  to  successfully  demonstrate  this 
technology  and  test  the  power  of  similarity  analysis  at  the  E12  exercise. 

An  important  extension  of  this  work  is  the  development  of  formal  procedures  to  determine  the 
value  of  information  (Vol)  in  context  of  the  current  operational  tempo  and  the  information’s 
content  and  source  reliability  [18].  All  information  should  not  be  weighted  equally. 
Additionally,  methods  need  to  be  developed  for  determining  appropriate  weighting  of  local 
attributes  and  the  modeling  of  linguistic  information. 


Appendix  A:  HDPT  Survey  Form 


HDPT  C4ISR  OTM  User  Survey 
C4ISR  MOD  On-Tlie-Move  Exercise  El 2 

Age: _ Years  MOS: _  Rank: _  Time  in  Service: _ Yrs 

What  is  your  current  Duty  Position? _ 

Are  you  Color  Blind? _  Computer  Use  Confidence:  1  2  3  4  5  (1-low;  5-high) 

Have  you  been  deployed? _  if  yes,  number  of  times _ 

Have  you  received  military  intelligence  training? _ 

If  yes,  what  courses  or  informal  training? 


1.  Please  circle  the  number  below  that  best  describes  how  well  you  were  able  to  use  this  tool. 


1 

!  2 

3 

i  4 

i  5  | 

VERY  POOR 

AVERAGE 

VERY  WELL 

2.  Please  circle  the  number  below  that  best  describes  your  awareness  of  the  evolving  High  Value 
Individual  (HVI)  intelligence  picture. 

|  1 

2 

3 

4 

|  5 

NOT  AT  ALL 
AWARE 

SOMEWHAT 

AWARE 

VERY  AWARE 

3.  Please  circle  the  number  that  best  describes  the  HDPT’s 

usefulness  in  tactical  deployments. 

|  1 

2 

3 

1  4 

1  5 

NOT  AT  ALL  SOMEWHAT  VERY  AWARE 

AWARE  AWARE 


Comments: 
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Background: 

Developmental  Reasoning 


Military  decision-making  difficulties  arise  from 
time  critical  analysis 

•  Of  data  that 


•  Has  increased  in  Amount  and  types  collected 

•  Has  become  inconsistent  and  incomplete 


The  Tactical  Information  and  Data  Fusion  Branch 
has  developed  a  computerized  Tool  to 

•  Assist  the  military  user  in  quickly  analyzing  large, 
inconsistent  data  sets 

•  Developed  to  utilize  existing  data  frameworks 
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0  RDEcnn/i  w  Background:  A  Solution 


The  Heterogeneous  Data  Proximity  Tool  (HDPT) 
Uses  Multi-Dimensional  Scaling  (MDS)  to  reduce 
high-dimensional  data  spaces,  such  human  terrain 
Data,  into  a  human-readable  visual  analytic 


Key  Features: 

•  Tolerant  of 
inconsistent  data 

•  3D  Similarity 
Display 

•  Interfaced  with 


DCGS-A  OWF 


•  Mobile  Input  Device 
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HDPT  Algorithms: 
Multi-Dimensional  Scaling 


Multidimensional  scaling  (MDS)  is  a  data  analysis  approach  used 
to  visually  interrogate  the  similarity  or  dissimilarity  between  the 
pair-wise  “distances”  among  a  given  set  of  objects. 
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This  example  from  Forrest 
Young’s  Understanding 
Multidimensional  Scaling  uses 
the  mileage  between  10 
American  cities,  shown  in  the 
table,  as  the  objective  similarity 
measure. 


The  associated  MDS  2-D 
visualization  output  would 
appear  something  like  that 
found  in  figure  1 .  Note:  the 
geometric  model  allows 
one  to  discern  the 
underlying  structure  and 
allow  human  interpretation. 
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Vs 


HDPT  Component 
Overview 


PostgrcSQL  DBS 


GlobalGraph  DB 


Apache  Tomcat  Web  Server 


Global  Graph  Web 

Application 


Comm  uni  cate  via  wab 

Svtr  i-rt  ij  S  i  >i  ( h'l  Jmd  Update) 

using  ISDN  Formal 


Apache  Tomcat  Web  Server 


Global  Graph 


Ozone  Widget  framework 


HDPT  Web  Application 


HDPT 


Communicate  urith  Rserve  , 
a  TCP/IP  server  allowing 
remote  <orm*(  bon  io  R, 
for  stylistic icakul  aliens. 


Statistics  Engine 
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HDPT  Visual  Tour 


*  <56  User  use<2  w  Password: 


□  Globa  l  Graph  Person  Search 


Hamn  5113.. 


Black 


Habib  Ala  A.. 


black 


AbuNawf... 


BLACK 


Rana  LiiEei. 


CRATJGE 


HAZARA 


Haroun  Sal 


GRANGE 


TAJIK 


N^iwaHad 


CRATJGE 
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Wall  31C 
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Black 


Nadi  Gauh 


GREEN 


Wa*  Muifa 


Kaiam  Ime. . 


CRINGE 


_ 

T’ASHTU 

/WOCH 

fc>A$HTO 


Sal-11  Mu  5' 


CRATJGE 
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TAJIK 
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black 
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GREEN 
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Raliq  Salt- . 


GREEN 
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San  Mis'id 


BLACK 


Ridba  Mali... 


BLACK 


Rasul  Ana .. 


BLACK 


Baniite'adl 


GREEN 


PASHTO 


Noslafa  Fa . 


Slia^raNa 


GREEN 


PASHTO 


PASHTO 


Name 


CdfCf 


MALE  A 


FEMJ 


Gala  Analysis  Global  Grapli:  impMocalhbslSDBD 


▼  ^rve; 

tfS  Q 


□  mDS  3D  Scatter  PW 


1. 


2. 


3. 


Menu  Bar  - 

Computer 
communications 
preferences,  data 
search,  and  plot 
controls 

Search  Window  - 

Modifiable  table  of 
entity  attributes 

Plot  Window  - 

Rotatable  3D 
similarity  visual 
analytic 
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HDPT 

C4ISR  OTM  CONOP 


Data  describing  and  individual  was 
collected  during  morning  and 
afternoon  exercise  events  ... 


Tribal  Affiliation 

Marital  Status 

Nationality 

Place  of  Birth 

EquipmentID 

VehiclelD 

Criminal  Record 

Education  Level 

Employment  Type 

Military  Record 

Religion 

Skill 

AddressID 

And  sent  to  the  TOC  /  DGCS-A 
Using  a  mobile  device  ... 


Where  it  was 
displayed  by 
HDPT  for 
analysis. 
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(J)  RDECOMJ ►  HDPT 

C4ISR  OTM  Scenario 


•  Each  entity  is  a  node  in  the  analytic 

•  Three  reference  sets  of  entities  for  HDPT: 

•  Red  or  Insurgent  — 

•  Green  or  Friendly 

•  Orange  or  Criminal - 

•  The  black  nodes  are  unknown  entities 


Each  entity  had  attributes 
as  shown 


•  Unknown  entity  data  was 
discovered  during  the  exercise 


Name 

SubID 

Location 

Org 

Bahij  As'adTawfeek 

39.98/-74.43 

Friendly 

Gen  TA 


FOB  Equip 


CR  Ed  Emp  MR  Rel  1 


28  M  Pastun  M  HN  BIA  Uniform  Burgundy  Luxury  Sbdan  No  High  WC  SS  id  ME  TSV 


As  attributes  were  discovered, 
unknown  entities  became 
similar  to  reference  nodes 


Entities  could  change  reference 
group  similarity  throughout  the 
exercise  day  * 

•  Changing  reference  example  Morning 


Exercise  Morning  Analytic 


From  analytics  such  as 
that  above  Soldiers 
were  asked  to  identify 
the  most  likely  reference 
group  membership  for 
each  unknown  entity 
node 


Group  Key 

C 

Criminal 

F 

Friendly 

1 

Insurgent 

U 

Unknown 

Strength  Rating 
1  -5 


Soldiers  rated  entity 
membership  using  a 
letter  identifying  each 
reference  group  or  a  “U 
for  undetermined  and  a 
Likert  scale  to  describe 
their  confidence  in  the 
determination 


Aali  Abu  Bakr  Karim  (S13) 

C4 

Abu  Navid  Sultan  (SI) 

Cl 

Amirah  Sani  El- Amin  (S12) 

U5 

Habib  Ala  Ahmed  (S5) 

F3 

Harun  Shahzad  El-Mofty  (S15) 

U5 

Hussain  Mansoor  El-Hashem  (S9) 

U5 

Ikram  I'timad  Abdullah  (S4) 

F3 

Khalilah  Qismat  Amirmoez  (S3) 

U3 

Rasul  Anass  Zaman  (S2) 

U5 

Rasul  Zayn  Mohammed  (S6) 

U3 

Ridha  Mahdi  El-Mofty  (S7) 

F2 

Saif-al-Din  Jinan  Hakim  (S14) 

13 

Sami  Mis'id  El-Ghazzawy  (S8) 

C3 

Yusuf  Mehmud  Samara  (S 1 1) 

U1 

Zaman  Noor  Hakim  (S10) 

U5 

Completed  rating  sheets 
were  compared  to 
ground  truth  entity 
membership  to 
determine  HDPT 
analysis  utility 
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Data  Analysis  Results  (All  Person  Attributes) 


Data  Analysis  Results  (Partial  Attributes) 


■ 

- 1 - 1 - 1 

Initial  Morning  Afternoon 

Analysis  Recording 

•  Soldier  ratings  were  gathered  over  4  exercise  days 

•  Initialized  nodes  •  Morning  event  nodes  •  Afternoon  event  nodes 

•  More  attributes  were  discovered  as  the  exercise  day  progressed 

•  Ground  truth  for  unknown  nodes  was  strictly  enforced 

•  The  entire  set  of  attributes  was  found  for  some  entities  (left  graph) 

•  A  partial  set  of  attributes  was  found  for  some  entities  (Right  graph) 


</> 

> 

nj 

c 

< 

*-> 

u 

<D 


o 

u 

T3 

<D 

N 

"ro 

E 


1 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.1 

0 


^Day  1 
^Day2 
^Day3 
^Day4 


Soldiers  determined  correct  group  affiliation  at  93%  accuracy. 
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Conclusion 


•  HDPT  provides  a  potentially  valuable  data 
analysis  advantage 

•  Soldiers  gave  high  ratings  for  tactical 
usefulness  and  usability 


•  HDPT  provides  easy-to-use  analytic  capabilities 
for  Soldiers  untrained  in  intelligence  analysis 


•  Results  support  further  development  and 
experimentation 


Plans  to  return  to  2013  C4ISR  OTM  field 


exercise  with  expanded  capabilities 
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